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ABSTRACT 

In principle, a network can transfer data at nearly the speed 
of light. Today’s Internet, however, is much slower: our 
measurements show that latencies are typically more than 
one, and often more than two orders of magnitude larger 
than the lower bound implied by the speed of light. Clos¬ 
ing this gap would not only add value to today’s Internet 
applications, but might also open the door to exciting new 
applications. Thus, we propose a grand challenge for the 
networking research community: building a speed-of-light 
Internet. Towards addressing this goal, we begin by inves¬ 
tigating the causes of latency inflation in the Internet across 
the network stack. Our analysis reveals that while protocol 
overheads, which have dominated the community’s atten¬ 
tion, are indeed important, infrastructural inefficiencies are 
a significant and under-explored problem. Thus, we propose 
a radical, yet surprisingly low-cost approach to mitigating 
latency inflation at the lowest layers and building a nearly 
speed-of-light Internet infrastructure. 

1. INTRODUCTION 

Reducing latency across the Internet is of immense value 
— measurements and analysis by Internet giants have shown 
that shaving a few hundred milliseconds from the time for a 
transaction can translate into millions of dollars. For Ama¬ 
zon, a 100ms latency penalty implies a 1% sales loss [42]; 
for Google, an additional delay of 400ms in search responses 
reduces search volume by 0.74%; and for Bing, 500ms of 
latency decreases revenue per user by 1.2% [20, 31]. Under¬ 
cutting a competitor’s latency by as little as 250ms is con¬ 
sidered a competitive advantage [12] in the industry. Even 
more crucially, these numbers underscore that latency is a 
key determinant of user experience. 

While latency reductions of a few hundred milliseconds 
are valuable, we take the position that the networking com¬ 
munity should pursue a much more ambitious goal: cutting 
Internet latencies to close to the limiting physical constraint, 
the speed of light, roughly one to two orders of magnitude 
faster than today. What would such a drastic reduction in In¬ 
ternet latency mean, and why is it worth pursuing? Beyond 
the obvious gains in performance and value for today’s ap¬ 
plications, such a technological leap could have truly trans¬ 
formative impact. A speed-of-light Internet may help realize 
the full potential of certain applications that have so far been 
limited to the laboratory such as tele-immersion. For some 


applications, such as massive multi-player online games, the 
size of the user community reachable within a latency bound 
plays an important role in user interest and adoption and, as 
we shall see later, linear decreases in communication latency 
result in super-linear growth in community size. Low laten¬ 
cies on the order of a few tens of milliseconds also open up 
the possibility of instant response , where users are unable 
to perceive any lag between requesting a page and seeing 
it rendered in their browsers. Such an elimination of wait 
time would be an important threshold in user experience. A 
speed-of-light Internet can also be expected to spur the de¬ 
velopment of new and creative applications. The creators of 
the Internet, after all, did not envision the myriad ways in 
which it is used today. 

But the Internet’s speed is quite far from the speed of light. 
As we show later, the fetch time from a set of generally well- 
connected clients for just the HTML document of the index 
pages of popular Web sites is, in the median, 35 times the 
round-trip speed-of-light latency. In the 80 th percentile it is 
more than 100 x slower. Given the promise a speed-of-light 
Internet holds, why are we so far from the speed of light? 

While ISPs compete primarily on the basis of peak band¬ 
width offered, bandwidth is not the issue. Bandwidth im¬ 
provements are also necessary, but bandwidth is no longer 
the bottleneck for a significant fraction of the population: 
for instance, the average Internet connection speed in the US 
is 11.5Mbps [18], while the effect of increasing bandwidth 
on page load time is small beyond as little as 5Mbps [39]. 
Projects like Google Fiber [4] and other fiber-to-the-home 
efforts by ISPs are further improving bandwidth. On the 
other hand, it has been noted in a variety of contexts from 
CPUs, to disks, to networks, that ‘latency lags bandwidth’, 
and reducing latency is a more difficult problem [50]. 

If bandwidth isn’t the culprit, then what is?! To identify 
the causes of latency inflation, we use two large datasets: 
2.9 million measurements 28,000 Web page downloads from 
186 clients to servers in 103 countries; and 2.4 million la¬ 
tency measurements between servers at a large CDN provider 
and end-users (1.7 million host pairs). We augment this data 
with IP geolocation data from five geolocation services. Our 
analysis of this data breaks down Internet latency inflation 
across the network stack, from the physical network infras¬ 
tructure through the transport layer. In line with the com¬ 
munity’s understanding, DNS, TCP’s three-way handshake, 
and TCP slow-start are all important factors in latency infla¬ 
tion. Also significant however, are the Internet’s infrastruc- 
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tural deficiencies. Apart from quantifying the contributions 
of various protocol factors to latency inflation, a key contri¬ 
bution of this work is putting latency inflation at the lowest 
layers in context — as we discuss in §4, the effect of im¬ 
provements at the lowest layers is multiplicative. We con¬ 
sider this an under-appreciated piece of the latency puzzle. 

Further, while networking research has largely taken in¬ 
frastructure as a given and focused on latency improvements 
in network protocols, we show that infrastructural impedi¬ 
ments to latency can be addressed surprisingly cheaply and 
with immediate payoff. We propose a radical approach to 
building a parallel, nearly speed-of-light Internet infrastruc¬ 
ture to augment our current high-capacity Internet backbone 
(§5). We analyze the cost, coverage, and effectiveness of our 
approach through a case study in designing such an infras¬ 
tructure for the contiguous United States. Our key finding is 
that a nearly speed-of-light Internet infrastructure connect¬ 
ing 85% of the US population can be built at the cost of a 
few hundred million dollars. In the context of Internet in¬ 
frastructure, where the cost of individual submarine cables 
can be many times larger [48], this is surprisingly cheap. 
Ironically, in this case, infrastructure might be an easier av¬ 
enue for rapid improvement than network protocols! 

However, as our measurements reveal, achieving speed- 
of-light latencies also depends on faster protocols. While 
the thrust of this paper is on tackling latency inflation at the 
lowest layers, recent advances in protocol research, which 
we discuss in §5.3, provide us with many possible solutions 
that may be put together to build a speed-of-light Internet. 

In summary, we propose a ‘speed-of-light Internet’ as a 
grand challenge for the networking community, quantify the 
factors that contribute to large latencies today, and propose 
a radical, but viable, approach to closing the latency gap. 

2. THE NEED FOR SPEED 

A speed-of-light Internet would be an advance with tremen¬ 
dous impact. It would enhance user satisfaction with Web 
applications, as well as voice and video communication. The 
gaming industry, where latencies larger than 50ms can hurt 
gameplay [49], would also benefit. But beyond the promise 
of these valuable improvements, a speed-of-light Internet 
could fundamentally transform the computing landscape. 
New applications. One of computing’s natural, yet unreal¬ 
ized goals is to create a convincing experience of joining two 
distant locations. In fact, tele-immersion topped the list of 
‘Killer Apps in the Gigabit Age’ themes in a Pew survey of 
1400+ experts 1 [22]. Applications like tele-immersion and 
remote collaborative music performance are hampered to¬ 
day by poor Internet latencies. For instance, latencies above 
50ms, make remote collaboration on music difficult [24], 
Convincing virtual reality immersion necessitates a latency 
of less than 20ms [61], and a similar limit likely applies to 
immersion in remote, real-world environments. Note that 
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VR immersion is vastly different from what is marketed as 
telepresence today, where communicating parties interact in 
a very static environment. A speed-of-light Internet could 
move such applications from their limited experimental scope, 
to ubiquity. And perhaps we will be surprised by the creative 
new applications that evolve in that environment 2 . 

Illusion of instant response. A speed-of-light Internet can 
realize the possibility of instant response. The human visual 
system cannot correctly order visual events separated by less 
than about 30ms [11], Thus, if responses over the Internet 
were received within 30ms of the requests, we would achieve 
the illusion of instant response 3 . A (perceived) zero wait¬ 
time for Internet services would greatly improve user expe¬ 
rience and allow for richer interaction. Immense resources, 
both computational and human, would become “instantly” 
available over a speed-of-light Internet. 

The Internet of Things. Tens of billions of devices (ex¬ 
cluding traditional personal computing devices) are expected 
to be on the Internet of Things by 2020 [35, 15]. For some of 
these devices, latency may not matter much - for a ‘smart’ 
thermostat to respond to temperature settings within several 
seconds is fine. However, other ‘smart things’ may be in¬ 
tended to facilitate active interaction with people, for ex¬ 
ample, shoppers using their wearable electronics to interact 
with merchandise that has smart tags. Such an augmented re¬ 
ality may depend on low-latency access to information over 
the Internet. Latencies close to the limits of human percep¬ 
tion would make these interactions seamless and natural. 
Super-linear community size. Many applications require 
that the connected users be reachable within a certain latency 
threshold, such as 30ms round-trip for instant response, or 
perhaps 50ms for a massive multi-player game. The value 
of low latency is magnified by the fact that the size of the 
available user community is a superlinear function of net¬ 
work speed. The area on the Earth’s surface reachable within 
a given latency grows nearly 4 quadratically in latency. Us¬ 
ing population density data 5 reveals somewhat slower, but 
still super-linear growth. We measured the number of people 
within a 30ms RTT from 200 World capital cities at various 
communication speeds. Fig. 1(a) shows the median (across 
cities) of the population reached. If Internet latencies were 
20x worse than c-latency (a;-axis=0.05c), we could reach 
7.5 million people “instantly”. A lOx latency improvement 
(a;-axis=0.5c) would increase that community size by 49x. 
Therefore, the value of latency improvement is magnified. 


2 “New capabilities emerge just by virtue of having smart people 

with access to state-of-the-art technology.” — Bob Kahn 
'This is a convenient benchmark number, but the exact number will 
vary depending on the scenario. For a 30ms response time, the In¬ 
ternet will actually need to be a little faster because of server-side 
request processing time, screen refresh delay, etc. And the ‘instant 
response’ threshold will differ somewhat for audio vs. visual appli¬ 
cations. 

4 Because the Earth is a sphere, not a plane. 

"Throughout, we use population estimates for 2010 [23]. 


2 







Communication speed Communication speed Global round-trip latency target (ms) 

(a) (b) (c) 

Figure Is The impact of communication speed on computing and people. With increasing communication speed: (a) the population within 30ms 
round-trip time grows super-linearly; (b) the number of locations (e.g. data centers or CDN nodes) needed for global 30 ms reachability from at least one 
location falls super-linearly; and (c) the tradeoff between the global latency target and the number of locations required to meet it improves. 


perhaps pushing some applications to reach critical mass. 
Cloud computing and thin clients. Another potential ef¬ 
fect of a speedier Internet is further centralization of com¬ 
pute resources. Google and VMware are already jointly work¬ 
ing towards the thin client model through virtualization [32], 
Currently, their Desktop-as-a-Service offering is targeted at 
businesses, with the customer centralizing most compute and 
data in a cluster, and deploying cheaper hardware as work¬ 
stations. A major difficulty with extending this model to per¬ 
sonal computing today is the much larger latency involved in 
reaching home users. Likewise, in the mobile space, there is 
interest in offloading some compute to the cloud, thereby 
exploiting data and computational resources unavailable on 
user devices [28]. As prior work [37] has argued, however, 
to achieve highly responsive performance from such appli¬ 
cations would today require the presence of a large number 
of data center facilities. With a speedier Internet, the ‘thin 
client’ model becomes plausible for both desktop and mo¬ 
bile computing with far fewer installations. For instance, if 
the Internet operated at half the speed of light, almost all of 
the contiguous US could be served instantly from just one 
location. Fig. 1(b) shows the number of locations needed for 
99% of the world’s population to be able to instantly reach 
at least one location — as we decrease Internet latency, the 
number of facilities required falls drastically, down to only 
6 locations with global speed-of-light connectivity. (These 
numbers were estimated using a heuristic placement algo¬ 
rithm and could possibly be improved upon.) This result is 
closely related to that in Fig. 1(a) — with increasing com¬ 
munication speed (which, given a latency bound, determines 
a reachable radius), the population reachable from a center 
grows super-linearly, and the number of centers needed to 
cover the entire population falls super-linearly. 

Better geolocation. As latency gets closer to the speed 
of light, latency-based geolocation gets better, and in the ex¬ 
treme case of exact c-latency, location can be precisely trian¬ 
gulated. While better geolocation provides benefits such as 
better targeting of services and matching with nearby servers, 
it also has other implications, such as for privacy. 

Don’t CDNs solve the latency problem? Content distri- 



Figure 2: Fetch time of just the HTML of the landing pages of popular 
Web sites in terms of inflation over the speed of light. 

bution networks cut latency by placing a large number of 
replicas of content across the globe, so that for most cus¬ 
tomers, some replica is nearby. However, this approach has 
its limitations. First, some resources simply cannot be repli¬ 
cated or moved, such as people — so CDNs are not rele¬ 
vant for all communication. Second, CDNs today are an ex¬ 
pensive option, available only to larger Internet companies. 
A speedier Internet would significantly cut costs for CDNs, 
putting them within reach of a larger spectrum of Web ser¬ 
vice providers. CDNs make a tradeoff between costs (de¬ 
termined, in part, by the number of infrastructure locations), 
and latency targets. For any latency target a CDN desires 
to achieve globally, given the Internet’s communication la¬ 
tency, a certain minimum number of locations are required. 
Speeding up the Internet improves this entire tradeoff curve. 
This improvement is shown in Fig. 1(c), where we estimate 
(using our random placement heuristic) the number of lo¬ 
cations required to achieve different latency targets for dif¬ 
ferent Internet communication speeds 6 : f, and c. As is 

clear from these results, while CDNs will still be necessary 
to hit global latency targets of a few tens of milliseconds, 
the amount of infrastructure they require to do so will fall 
drastically with a speedier Internet. 

3. THE INTERNET IS TOO SLOW 

We fetched just the HTML for the landing pages of 28,000 

'’Per our measurements in §3, ^ is close to the median speed of 
fetching just the HTML for the landing pages of popular websites 
today, and | is close to the median ping speed. 
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popular Web sites from 186 PlanetLab locations using cURL [1], 
We pooled Alexa’s [13] top 500 Web sites from each of 103 
countries and used the unique URLs. We followed redirects 
on each URL, and recorded the final URL for use in our 
measurements. In our experiments, we ignored any URLs 
that still caused redirects. The results presented here exclude 
data for the few hundreds of Web sites in our sample that use 
TLS/SSL; an analysis accounting for the latency cost of es¬ 
tablishing secure connections is left to future work. 

For each connection, we geolocated the Web server us¬ 
ing commercial geolocation services, and computed the time 
it would take for light to travel round-trip along the short¬ 
est path between the same end-points, i.e., the c-latency 7 . 
Henceforth, we refer to the ratio of the fetch time to c-latency 
as the Internet’s latency inflation. Fig. 2 shows the CDF 
of inflation over 2.9 million connections. The HTML fetch 
time is, in the median, 35.4x the c-latency, while the 80 th 
percentile exceeds 100 x. Thus, the Internet is typically more 
than an order of magnitude slower than the speed of light, 
and often two orders of magnitude slower. We note that 
PlanetLab nodes are generally well-connected, and latency 
can be expected to be poorer from the network’s true edge. 

4. WHY IS THE INTERNET SO SLOW? 

To identify the causes of Internet latency inflation, we 
break down the fetch time across layers, from inflation in the 
physical path followed by packets to the TCP transfer time. 

We first describe the methodology (§4. 1) and an overview of 
the results (§4.2). Next, we discuss the robustness of our re¬ 
sults to IP geolocation errors (§4.3), and consistency across 
page fetch sizes (§4.4) and geographies (§4.5). In §4.6, we 
investigate the role of congestion, and lastly, in §4.7, the im¬ 
pact of infrastructural deficiencies. 

4.1 Methodology 

We use cURL to obtain the time for DNS resolution, TCP 
handshake, TCP data transfer, and total fetch time for each 
connection. The TCP handshake is measured as the time be¬ 
tween cURL sending the SYN and receiving the SYN-ACK. 
The TCP transfer time is measured as the time from cURL’s 
receipt of the first byte to the receipt of the last byte. We sep¬ 
arately account for the time between cURL sending the data 
request and the receipt of the first byte as ‘request-response’ 
time; this typically comprises one RTT and any processing 
time at the Web server. For each connection, we also run a 
traceroute from the client PlanetLab node to the Web server. 

We then geolocate each router in the traceroute, and connect 
successive routers with the shortest paths on the Earth’s sur¬ 
face as an approximation for the route the packets follow. We 
compute the roundtrip latency at the speed of light in fiber 
along this approximate path, and refer to it as the ‘router- 
path latency’. From each client, we also run 30 successive 

7 We have ground-truth geolocation for PlanetLab nodes — while 
the PlanetLab API yields incorrect locations for some nodes, these 
are easy to identify and remove based on simple latency tests. 



Figure 3: Various components of latency inflation. The median is 
marked on each curve for sake of clarity of the legend. 

pings to each Web server, and record the minimum and me¬ 
dian across these ping times. We normalize each of these 
latency components by the c-latency between the respective 
connection’s end-points. 

4.2 Overview of results 

Fig. 3 shows the results for all 2.9 million connections 8 . It 
is unsurprising that DNS resolutions are faster than c-latency 
8% of the time — in these cases, the server happens to be 
farther than the DNS resolver. (The DNS curve is clipped 
at the left to more clearly display the other results.) In the 
median, DNS resolutions are 7.4 x inflated over c-latency. 

The TCP transfer time shows significant inflation —10.2 x 
in the median. With most pages being tens of KBs, band¬ 
width is not the problem, but TCP’s slow start causes even 
small data transfers to require several RTTs. 20% of all 
pages have transfer times less than the c-latency — this is 
due to all the data being received in the first TCP window. 
(Recall that transfer time is the time between cURL receiv¬ 
ing the first and the last bytes.) The TCP handshake (count¬ 
ing only the SYN and SYN-ACK) and the minimum ping 
time are 3.4x and 3.2x inflated in the median. 

The request-response time is 6.6x inflated in the median, 
i.e., roughly twice the median round-trip time. However, 
25% of the connections use less than 10ms of server pro¬ 
cessing time (estimated by subtracting one RTT from the 
request-response time). 

It is worth noting that the medians of inflation in DNS 
time, TCP handshake time, request-response time, and TCP 
transfer time add up to 27.6x, in comparison to the mea¬ 
sured median total time of 35.4x. We should expect such a 
discrepancy because of the distributions being tail heavy. 

4.3 Impact of IP geolocation errors 

While we cull data with obvious anomalies arising from 
geolocation errors (such as when the minimum ping time is 
smaller than the c-latency computed based on IP geoloca¬ 
tion), less obvious errors could impact our results. Obtain- 

8 Any connections where our data showed obvious anomalies, such 
as c-latency being larger than the minimum ping time due to ge¬ 
olocation errors, were weeded out; 2.9 million connections survive 
such checks. 
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(a) Router-path latency inflation (b) Minimum ping latency inflation (c) Total fetch time inflation 

Figure 4: Results for median, 80 th -percentile, and 95 th -percentile of inflation in several metrics, using 5 different commercial geolocation databases 
as well as their majority vote (MV). 





Figure 5: Internet latency inflation measured across page sizes in our measurements: (a) the distribution of Web page sizes in our traces; (b) inflation 
in minimum ping time, TCP transfer time, and total time as a function of page size; and (c) various components of latency inflation across pages within 
10% of the median page size. 


ing ground truth information for the large IP space under 
consideration appears infeasible. Thus, we focused our ef¬ 
fort on comparing the results we obtained by using 5 differ¬ 
ent commercial IP geolocation services, as well as a location 
computed as their majority vote (MV). We computed latency 
inflation in router-path, minimum ping, and total time using 
each of these 6 sets of IP geolocations. Fig 4 shows the com¬ 
parison; as we might expect, router-path latency (Fig. 4(a)) 
is most susceptible to differences in IP geolocation — the 
result there depends on geolocating not only the Web server, 
but also each router along the path. Even so, all 6 median in¬ 
flation values are in the 2-2.4x range. Differences in results 
for minimum ping time (Fig. 4(b)) and total time (Fig. 4(c)) 
are much smaller. Even the 95 th -percentile values for infla¬ 
tion in minimum ping time all lie within 10.1-11.Ox, while 
the medians lie within 3.2-3.3x. The results for median in¬ 
flation in total time all lie between 35.4-37.6x, but varia¬ 
tion at the higher percentiles is larger. With the exception of 
Fig. 4, we use the majority vote geolocation throughout. 

4.4 Results across page sizes 

While we only fetch the HTML for the landing pages of 
Web sites in our experiments, some of these are still larger 
than 1MB. However, as Fig. 5(a) shows, most pages are 
much smaller, with the median being 47KB. To analyze vari¬ 
ations in our results across page sizes, we binned pages into 
1KB buckets, and computed the median inflation for each 
latency component across each bucket. Fig. 5(b) shows the 
median inflation in ping time, TCP transfer time, and to¬ 


tal time across different page sizes. The median inflation in 
minimum ping time shows little variation across page sizes, 
as one might expect. Inflation in TCP transfer time increases 
over page sizes in an expected fashion, also causing an in¬ 
crease in total fetch time. 

We also examine latency inflation in a narrow range of 
Web page sizes around the median, using pages within 10% 
of the median size of 47KB. These pages comprise roughly 
6% of our dataset. The results of this analysis are shown 
in Fig. 5(c), and are similar to the overall results in Fig. 3, 
with expected differences in the transfer time and total time 
curves. The medians for various components of latency in¬ 
flation are all within 3% of the results in Fig. 3, except request- 
response time where the median is 21% larger for this set. 

4.5 Results across geographies 

We fetch pages in 103 countries from 186 unique Plan- 
etLab locations, leading to a wide spread in the pairwise 
c-latencies observed across these connections. This varia¬ 
tion is captured in Fig. 6(a), which shows the distribution of 
traces across different c-latencies. (c-latencies were binned 
into 1ms bins for this analysis.) The shape of the curve is 
largely a result of the Earth’s geography and the distribution 
of our PlanetLab clients. The median c-latency is 43ms. In a 
manner similar to our analysis across page sizes, we also an¬ 
alyzed latency inflation across c-latencies. Fig. 6(b) shows 
the median inflation in router-path latency, minimum ping 
time, and total time. As one might expect, latency inflation 
is higher for small c-latencies. An interesting feature of these 
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Figure 6: Internet latency inflation measured across c-latencies and geographies: (a) the distribution of c-latencies across host-pairs in our traces; (b) 
inflation in router-path latency, minimum ping time, and total time as a function of c-latency; and (c) inflation in router-path latency, minimum ping time, 
DNS time, and total time across 8 countries. 


results is the inflation bump around a c-latency of 30ms. It 
turns out that countries such as Portugal, Iran, Ireland, Ice¬ 
land, and Ecuador, connectivity to which may be more cir¬ 
cuitous than average, are over-represented at these distances 
in our data. For instance, c-latencies from the Eastern US to 
Portugal are in the 30ms vicinity, but all transatlantic con¬ 
nectivity hits Northern Europe, from where routes may go 
through the ocean or land Southward to Portugal, thus in¬ 
curring significant path ‘stretch’. That the differences are 
largely due to inflation at the lowest layers is also borne out 
by the inflation in minimum ping and total time following 
the inflation in the router-path latency. 

For a fairer comparison across geographies, we selected 
28 PlanetLab hosts such that no two were within 5 degrees 
of longitude of each other. Then we looked at requests from 
these PlanetLab clients to Web servers in each country. Fig. 6(c) 
shows the median inflation in router-path latency, minimum 
ping time, DNS, and total time across each of the 8 coun¬ 
tries for which we had 10,000+ connections. The median 
c-latencies (not shown in Fig. 6(c)) from these selected Plan¬ 
etLab hosts to each these 8 countries all lie in the 47-54ms 
range, with the exception of China (59ms). This is likely at¬ 
tributable to the distribution of PlanetLab hosts — no Plan¬ 
etLab hosts were available to us between longitudes 38°E- 
100°E, thus placing China further away from our hosts on 
average than other nations. Router-path latencies are fairly 
consistent across geographies, with the exception of fetches 
from China, which is also much worse than the others for 
each of minimum ping time, DNS, and total time, for reasons 
that are not clear to us 9 . Across the other 7 countries, median 
inflation ranges between 2.8-3.6x for minimum ping time, 
6-7. 7x for DNS, and 26.1-32. 5x for total time. 

4.6 The role of congestion 

Fig. 3 and Fig. 5(c) show that TCP transfer time is more 
than 10 x inflated over c-latency. It is worth considering 
whether packet losses or large packet delays and delay vari¬ 
ations are to blame for poor TCP performance. 

In addition to fetching the HTML for the landing page, 
for each connection, we also sent 30 pings from the client to 

9 Ahem ... (Great) ... ahem ... (Firewall)? 


the server’s address. We found that variation in ping times 
is small: the 2" d -longest ping time is only 1.1% larger than 
the minimum ping time in the median. While pings (using 
ICMP) might use queues separate from Web traffic, even the 
TCP handshake time is only 1.6% larger than the minimum 
ping time in the median. We also used tcpdump [10] at Plan¬ 
etLab clients to log packet arrival times from the servers, and 
analyzed the inter-arrival gaps between packets. More than 
92% of the connections we made experienced no packet loss 
(estimated as packets re-ordered by more than 3ms). 

For a closer look at congestion in true end-user environ¬ 
ments (as opposed to PlanetLab), we examined RTTs in a 
sample of TCP connection handshakes between the servers 
of a large CDN and clients (end users) over a 24-hour time 
period, passively logged at the CDN. (Most routes to pop¬ 
ular prefixes are unlikely to change at this time-scale in the 
Internet [54].) We exclude server-client pairs with minimum 
latencies of less than 3ms — ‘clients’ in this latency range 
are often proxy servers in a data center or colocation facility 
rather than our intended end-users. 

To evaluate the impact of congestion, we examine our data 
for both variations across time-of-day (perhaps latencies are, 
as a whole, significantly larger in peak traffic hours), and 
within short periods of time for the same server-client pairs 
(perhaps transient congestion for individual connections is 
a significant problem). Thus, we discard server-client pairs 
that do not have repeat measurements. For ease of analy¬ 
sis over time-of-day, we only look at pairs within the same 
country. Server locations were provided to us by the CDN, 
and clients were geolocated using a commercial geoloca¬ 
tion service. We include here results for a few geographies 
that have a large number of measurements after these restric¬ 
tions. We bin all RTT measurements into 12 2-hour periods 
and produce results aggregated over these bins separately for 
each country. 

Time-of-day latency variations across bins: We selected 
server-client pairs that have at least one RTT measurement 
in each of the twelve bins. For pairs with multiple RTTs 
within a bin, we use the median RTT as representative, dis¬ 
carding other measurements. This leaves us with the same 
number of samples between the same host-pairs in all bins. 
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Fig. 7(a) and Fig. 7(b) show the median and the 90 th per¬ 
centile of RTTs in each 2-hour bin for each of 5 timezones. 
For the United States (US), we show only data for the cen¬ 
tral (CST) and eastern (EST) timezones, but the results are 
similar for the rest 10 . Median latency across our aggregate 
varies little across the day, most timezones seeing no more 
than 3ms of variation, except Great Britain, where the max¬ 
imum latency difference is 7.35ms. The 90 th percentile in 
each bin (Fig. 7(b) shows similar trends, although with larger 
variations. Again, in Great Britain, RTTs are higher in the 
evening. (We checked that results for a different 24-hour 
period look similar.) It is thus possible that congestion is 
in play there, affecting network-wide latencies. However, 
across other timezones, we see no such effect. 

Latency variations within bins: To investigate variations 
within bins, we do not limit ourselves to measurements across 
the same set of host-pairs across all bins. However, within 
each bin, only data from host-pairs with multiple measure¬ 
ments inside that time period is included. For each host- 
pair in each bin, we calculate the maximum change in RTT 
( A max ) - the difference between the maximum and mini¬ 
mum RTT between the host-pair in that time period. We then 
compute the median A max across host-pairs within each bin. 
Fig. 7(c) shows the results: the variation within bins is a bit 
larger than variations across median latencies across the day. 
For example, for US (CST), the median A max is as large as 
9ms in the peak hours. That A rnax also shows broadly sim¬ 
ilar time-of-day trends to median latency is not surprising. 
GB continues to show exceptionally large latency variations, 
with a A max ~25ms at the peak, and also large variations 
across the day. 

To summarize, even in the PlanetLab context, where la¬ 
tency variations and congestion are minimal, Internet laten¬ 
cies show large inflation. In end-user environments, network¬ 
wide latency increases in peak hours were largely limited in 
our measurements to one geography (Great Britain). How¬ 
ever, individual flows may occasionally experience a few ad¬ 
ditional milliseconds of latency. 

4.7 Path inflation 

Fig. 3 shows that in the median, the router-path is only 2x 
inflated. The long tail is, in part, explained by ‘hairpinning’, 
i. e ., packets between nearby end-points traversing circuitous 
routes across the globe. For instance, in some cases, packets 
between end-points in Eastern China and Taiwan were seen 
in our traces traveling first to California. Note that 1.5x in¬ 
flation would occur even along the shortest path along the 
Earth’s surface because the speed of light in fiber is roughly 
2/3 rd the speed of light in vacuum. In that light, 2x may 
appear small. But as we discuss below, our router-path es¬ 
timate is optimistic, and the lower layers play a significant 

"’The timezone classification is based on the location of the client; 
servers associated with these measurements can be anywhere in the 
US and not necessarily restricted to the same timezone as that of 
the clients. 


role in overall inflation. 

We see some separation between the minimum ping time 
and the router-path latency. This gap may be explained by 
two factors: (a) traceroute often does not yield responses 
from all the routers on the path, in which case we essentially 
see artificially shorter paths — our computation simply as¬ 
sumes that there is a direct connection between each pair of 
successive replying routers; and (b) even between successive 
routers, the physical path may be longer than the shortest 
arc along the Earth’s surface. We investigate the latter as¬ 
pect using data from three research networks: Internet2 [7], 
ESnet [2] n , and GEANT 12 . We obtained point-to-point fiber 
lengths for these networks and ran an all pairs shortest paths 
computation on each fiber-length annotated network map to 
calculate end-to-end fiber distances between all pairs of end¬ 
points in each network. We also calculated the shortest dis¬ 
tance along the Earth’s surface between each pair of end¬ 
points, and obtained the road distances for comparison us¬ 
ing the Google Maps API [6], Fig. 8 shows the inflation 
in fiber lengths and road distances compared to the short¬ 
est distance. Road distances are close to shortest distances, 
while fiber lengths are significantly larger and have a long 
tail. The median inflation in the three networks, after ac¬ 
counting for the lower speed of light in fiber (Fig. 8 does not 
include this adjustment), is 2.6x (Intemet2), 2.7x (ESnet), 
and 3.6x (GEANT). Thus, infrastructural inflation (which 
includes routing sub-optimalities and inflation of end-to-end 
fiber-distances over geodistance) is likely to be larger than 
the optimistic estimate from router-path latency (2x), bring¬ 
ing it closer to the inflation in minimum ping latency (3.2x). 

Putting inflation at lower layers in perspective: As Fig. 3 
shows, DNS resolution (7.4 x inflated over c-latency), TCP 
handshake (3. 4x), request-response time (6.6x), and TCP 
transfer (10.2x), all contribute to a total time inflation of 
35.4x. With these numbers, it may be tempting to dismiss 
the 3.2 x inflation in the median ping time. But this would 
be incorrect because lower-layer inflation, embodied in RTT, 
has a multiplicative effect on each of DNS, TCP handshake, 
request-response, and TCP transfer time. The total time for 
a page fetch can be broken down roughly (ignoring minor 
factors like the client stack) as: 

Ttotal — TdNS T ^handshake “f -Fr equest 

H - T server p roc H - T res p 0nse Tt rans f er 

If we changed the network’s RTTs as a whole by a factor 
of x, everything on the RHS except the server processing 
time (which can be made quite small in practice) changes 
by a factor of x (to an approximation; TCP transfer time’s 
dependence on RTTs is a bit more complex), thus changing 
Ttot.ai by approximately a factor of x as well. 

1 'Dhruv Diddi helped process the ESnet data. 
l2 Data on fiber mileages from GEANT[3], the high-speed pan- 
European research and education network, was obtained through 
personal communication with Xavier Martins-Rivas, DANTE. 
DANTE is the project coordinator and operator of GEANT. 


7 






Time (in UTC) 


Time (in UTC) 
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Figure 7: Variations in latencies of client-server pairs grouped into 2-hr windows in different geographic regions: (a) Medians of RTTs of client-server 
pairs with measurements in each 2-hr window; (b) 90 th percentiles of RTTs the same set of client-set pairs; and (c) medians of maximum change in RTTs 
(max - min) in repeat measurements within each time window. 





Figure 8: Compared to the shortest distance along the Earth’s surface, there is significantly more inflation in fiber lengths than in road distances in all 
three networks (a) Internet2; (b) ESnet; and (c) GEANT. 


What if there was no inflation in the lower layers, i.e., 
RTTs were the same as c-latencies? For an approximate an¬ 
swer, we can normalize inflation in DNS, TCP handshake, 
request-response (excluding the server processing time, i.e., 
only the RTT) and TCP transfer time to that in the minimum 
ping time. Normalized by the median inflation in ping time 
(3.2x), these numbers are 2.3x (DNS), l.lx (TCP hand¬ 
shake), 1.0 x (request-response, excluding server processing 
time), and 3.2x (TCP transfer) respectively. The inflation in 
ping time is at par with the largest of these other (normal¬ 
ized) factors! Further, if, for example, TCP transfer could be 
optimized such that it happens within an RTT, the Internet 
would still be worse than ~25x slower than the c-latency in 
the median, but if we could cut inflation at the lower layers 
from 3.2x to close to lx, even if we made no transport pro¬ 
tocol improvements, we would get to around ~12x. These 
numbers are, of course, approximate assessments, the larger 
point being that the contribution of inflation at lower layers 
is multiplicative. Thus, inflation at the lower layers plays a 
big role in Internet latency inflation, and getting to a speed- 
of-light Internet requires both infrastructural improvements 
and protocol advances. 

5. FAST-FORWARD TO THE FUTURE 

In line with the community’s understanding, our measure¬ 
ments affirm that TCP transfer and DNS resolution are im¬ 
portant factors causing latency inflation. However, building 
a speed-of-light Internet requires addressing not only infla¬ 


tion due to protocols, but also that stemming from the In¬ 
ternet’s infrastructure. Is this then, a lost cause, as infras¬ 
tructural problems often are deemed to be? No! In fact, we 
present here a surprisingly low-cost solution to the infras¬ 
tructural problem. 

5.1 A parallel low-latency infrastructure 

The approach we propose is to build a “parallel Inter¬ 
net” to move traffic along nearly the shortest paths on the 
Earth’s surface at nearly the speed of light in vacuum. How 
might we build such an alternate Internet infrastructure? In 
addressing this question, we draw inspiration from the in¬ 
dustry which perhaps places the highest premium on mil¬ 
liseconds today: high frequency trading. The HFT industry 
has already demonstrated the plausibility of operating long¬ 
distance links at nearly the speed of light in vacuum. In 
the quest to cut latency between the New York and Chicago 
stock exchanges, several iterations of this connection have 
been built, aimed at successively improving latency by just 
a few milliseconds at the expense of hundreds of millions of 
dollars [41], In the mid-1980s, the round-trip latency was 
14.5ms. This was cut to 13.1ms by 2010 by shortening the 
physical fiber route. In 2012 however, the speed of light 
in fiber was declared too slow, microwave communication 
cut round-trip latency to 9ms, and later down to 8.5ms [27, 
16]. The c-latency, i.e., the round-trip travel time between 
the same two locations along the shortest path on the Earth’s 
surface at the speed of light in vacuum, is only 0.6ms less. A 
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similar race is underway along multiple segments in Europe, 
including London-Frankfurt [9]. 

Today at least 15 such microwave networks connect the 
Chicago and New York stock exchanges [41], These net¬ 
works are built using a series of microwave repeaters mounted 
on towers along roughly the shortest path along the Earth’s 
surface between the two cities. The ‘building block’ is a mi¬ 
crowave repeater mounted on a tower which can span 70km 13 
between towers at a data rate of 400Mbps, with the turnkey 
installation cost of one repeater ranging from $100,000-250,000 
(“including equipment, path engineering, site and path sur¬ 
veys, frequency coordination and licensing, tower work, in¬ 
stallation, and commissioning” [41]), with an operating cost 
(including a tower lease, power, maintenance, network oper¬ 
ating centers, etc.) of $38,000 per year. The typical turnkey 
installation costs are closer to $100,000, but hit the high end 
estimate in the Chicago-New York segment due to competi¬ 
tion for towers and spectrum. 14 

In short, our proposal is to emulate this idea on a wide 
scale to build a “c-network”. We could connect pairs of 
cities using a series of microwave towers. Henceforth, we re¬ 
fer to such a city-to-city microwave connection as an ‘edge’, 
with a c-network being a network of such edges. An existing 
Chicago-New York edge, for instance, uses 18 towers in this 
manner [41], At its end-point cities, each edge would con¬ 
nect to routing infrastructure, thus becoming part of a net¬ 
work with wide geographic coverage. Of course, fashioning 
a practical solution from this approach requires careful deci¬ 
sions about the extent of such infrastructure’s geographical 
coverage, and its capacity and usage. In the following, we 
address these design decisions first in the abstract, and then 
with a case study of such a proposal limited to the US. 

Geographical coverage: Building a ubiquitous speed-of- 
light Internet with today’s technology is likely to be cost- 
prohibitive. A more reasonable approach is to connect cen¬ 
ters of dense population into a c-network, and use traditional 
connectivity such as fiber to reach areas up to 100km or so 
from these centers. In the typical case, the two communicat¬ 
ing end-points would use traditional Internet infrastructure 
to reach their closest c-networked centers, from where traf¬ 
fic would be carried over the c-network. Using fiber over 
100km incurs an additional |ms of latency round-trip in 
comparison with c-connectivity. Thus, even if the physical 
paths in these final 100km of geo-distance are somewhat cir¬ 
cuitous, we can expect the total latency overhead to be lim¬ 
ited to 2-3ms for such an end-to-end connection. The goal 
should be to cover the large distances at nearly c. 

Obviously, some scenarios pose difficulties for such a de¬ 
ployment. Building trans-Atlantic microwave connectivity, 
for example, will require further innovation, although the 


' ’This number depends on terrain, but at least one network in the 
Chicago-New York segment uses towers close to this distance from 
each other [41]. 

14 Personal correspondence with Gregory Laughlin and Anthony 
Aguirre, authors of [41], 


HFT industry is already considering the possibility of us¬ 
ing weather balloons to overcome that challenge [52]. Over 
time, other technologies which operate at nearly the speed 
of light in vacuum (such as the under-development “hollow 
fiber” [29]) may come to fruition, but microwave appears 
presently to be the only mature, cheap solution. 

Capacity and usage: Matching the present Internet’s band¬ 
width in a parallel speed-of-light infrastructure would be ex¬ 
orbitantly expensive, perhaps even impossible — there might 
not even be enough wireless spectrum to be able to accom¬ 
plish this with today’s technology. However, most traffic 
on the Internet is not latency sensitive. For example, video 
streaming, hie sharing, and software downloads comprise an 
overwhelming fraction of the traffic on the Internet. Further, 
even most interactive Web traffic consists of small requests 
(few KB) which fetch responses larger by two to three orders 
of magnitude. While small responses may be accommodated 
over the c-network, larger responses could be sent over tra¬ 
ditional connectivity. If protocol overheads were negligible, 
this would still achieve % -connectivity overall. Thus, target¬ 
ing 1% of the latency sensitive Web traffic’s capacity needs 
would be a reasonable goal. 

Admittedly, some latency-sensitive applications we dis¬ 
cuss in §2 (such as tele-immersion) depend on both high 
bandwidth and low latency. While our present proposal may 
be inadequate for such applications at scale due to limited 
bandwidth, it still serves other applications such as instant 
access. We consider this work a start in this direction. 

In some sense, this infrastructure may be easier to make 
progress on than many protocol changes, which depend on 
consensus among several stakeholders. It also has a healthy 
incremental deployment potential. An ISP might start by 
selling a low-latency transit service to other ISPs on a few 
critical routes. It might also be possible to market directly to 
end-users by making use of tunneling in a manner suggested 
by a recent research proposal [51], where a user sends their 
packets first to an IP address connected to the c-network, 
which tunnels them to the exit from the c-network which is 
nearest to the packets’ final destination. (In this sense, the 
c-network uses “cold potato” routing.) 

In the following section, we analyze the coverage, capac¬ 
ity, and cost of a nearly speed-of-light network across the 
contiguous United States. 

5.2 A c-ISP in the US 

Geographical coverage: We focus on connecting only the 
200 most populous cities in the contiguous United States 
with each other over this infrastructure. Further, we coalesce 
suburbs and cities in close vicinity of each other (within 50 
kilometers), ending up with 120 population centers. Based 
on population data for 2010 [23], we calculate that ~258 
million people comprising ~85% of the US population live 
within 100 kilometers of these 120 population centers. 

Capacity and usage : Cisco’s forecast for “Consumer Inter- 
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Figure 9 1 A nearly speed-of-light network built across the United States over microwave towers. The gray lines show the logical city-to-city direct 
microwave connections, which are then realized using the shortest possible chain of microwave towers between each pair of cities connected. Each tower 
is marked by an individual marker. No two towers on a path are farther than 70 kilometers. 


net Traffic” for 2015 in North America 15 is ~ 1 1,500 PB/month, 
i.e., ~4.5TBps [25]. Cisco also estimates that 78.5% of 
Global IP traffic is file sharing and video, the rest (22.5%) 
being classified as “Web / Data” traffic 1 ' 1 . This leaves a 
ceiling of lTBps for interactive Web traffic, 1% of which 
is 80Gbps. Cisco’s report also claims that Web traffic is 
“spread evenly throughout the day”, unlike video, which has 
a “prime time” in each geography. Thus, we target 80Gbps 
as our c-ISP’s capacity. 

In building the c-ISP, our objective is to ensure that end- 
to-end paths between all pairs of locations are as close to the 
shortest paths along the Earth’s surface as possible (i.e., path 
stretch close to 1). Our budget is limited largely by the total 
number of microwave towers needed. 

While, tower leasing companies advertise a willingness to 
build towers for long-term leases, we restrict ourselves to 
using towers that are registered in FCC’s ‘Antenna Structure 
Registration’ database [33] 17 , with a status of ‘Constructed’. 


l5 An equivalent number for the contiguous US was not available. 
We note that the US population is only two-thirds of that of North 
America, but continue to use Cisco’s North America traffic estimate 
without any proportional adjustment as a conservative upper bound 
for our analysis. 

16 It is worth noting that traffic such as software downloads (except 
when over P2P software) is included in the this category, but is not 
latency sensitive. 

l7 Note that FCC only requires registrations for towers about 200 


While many of these towers may not be available for leasing, 
their existence, at the least, indicates the suitability of their 
near vicinity for the construction of other towers. We thus 
focus on using these towers as the foundation of our US- 
wide microwave network. 

We used an intuitive heuristic to design the network, start¬ 
ing with a minimum cost spanning tree (with the cost being 
number of towers needed to connect a pair of cities), and 
repeatedly augmenting this network with the edge that min¬ 
imizes the 95 th percentile stretch across all bytes that would 
move through the network. In the traffic matrix we used, 
traffic between each pair of cities is proportional to the prod¬ 
uct of their populations. We normalized the traffic matrix 
such that the total traffic hit our 80Gbps capacity require¬ 
ment. Of course, some edges end up with more load than 
their 400Mbps capacity — we replicate each edge enough 
times not only to accommodate its load, but also to operate 
at 50% utilization to allow for traffic variations. (Replicating 
an edge once implies using a parallel series of towers along 
the route, thus doubling the tower-count.) Our designed net¬ 
work connects 300 city-pairs with microwave edges built 
over a total of 2526 towers (after accounting for any nec¬ 
essary replications), and is shown in Fig. 9. 

Fig, 10 shows the path stretch incurred by bytes across 

feet, or otherwise, in sensitive locations such as near airports. En¬ 
forcement of registrations is also difficult, so there are certainly 
more towers suitable for antennas than in this database. 
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Figure 10: The path stretch in our designed network is small. 

the entire network. We show the stretch results for two sce¬ 
narios: one where towers are ubiquitous, i. e., edges between 
cities are along the geodesics; and another using only the 
towers in the FCC database. The median and 90 4,t -percentile 
stretch numbers are 1.03 x and 1.09x (with towers being 
ubiquitous) and 1.08x and 1.15x (with only the towers in 
FCC’s data). Thus, most bytes are carried at close to c- 
latency. It is also worth noting that our stretch results are 
not too sensitive to the 70km range of a microwave repeater, 
although, obviously, the number of towers required increases 
as the range decreases. Limiting the range to 50km results in 
a network with 3320 towers and median and 90 4/t -percentile 
stretch values of l.lx and 1.18x, while at 40km, we need 
4296 towers for stretch values of 1.12x and 1.41 x. When 
the range is limited to 30km, some cities are disconnected 
from the network. 

At a $100,000 installation cost per tower, this network 
would cost $253 million, with a $96 million per year op¬ 
erating cost. Amortized over 5 years, the yearly cost would 
be ~$ 1 17 million. In comparison, the Internet2 predecessor, 
Abilene, cost $500 million (1998, unadjusted) [26]. That 
other ambitious projects with substantially larger price-tags 
are making progress is also encouraging; among these are 
OneWeb’s plan to expand the reach of the Internet using 
a total of 648 satellites in low-Earth orbit at an estimated 
cost of $1.5-2 billion [36], and a broadly similar effort from 
WorldVu, cost estimates for which range from $10 billion 
to 15 billion [36, 19]. Arctic Fiber is spending $850 mil¬ 
lion to build a submarine cable through the Arctic, focused 
primarily on shortening the London-Tokyo route [48], 

Obviously, our analysis here is coarse, with many oppor¬ 
tunities for refinement of the network’s design. However, 
the larger point is that a nearly speed-of-light infrastructure 
is feasible to build today relatively cheaply. 

It is worth mentioning that microwave is not the most reli¬ 
able medium in inclement weather. However, in the Chicago- 
New York segment, which is not particularly known for clear 
weather, the operational networks claim a 95% uptime, with 
even low estimates of availability being over 85% [21], Fur¬ 
ther, we continue to have backup connectivity over fiber, and 
even if a microwave edge connecting two cities is unavail¬ 
able, the latency gains from the rest of the infrastructure con¬ 
tinue to provide value. 


5.3 A one RTT transaction protocol 

As we note in §4, while eliminating infrastructural latency 
inflation will speed up the Internet by more than 3x, with¬ 
out any progress on transport protocols, this may still leave 
us more than 10 x away from c-latency in the typical case. 
Thus, while the main contributions of this work are our mea¬ 
surements and analysis highlighting latency inflation due to 
infrastructural inefficiencies, and our proposal for a nearly 
speed-of-light physical infrastructure, we also discuss here 
the possibility of building a one RTT transaction protocol 
for the Internet by putting together various protocol advance¬ 
ments the community’s research has produced. 

DNS: One possibility to speed up DNS lookups is simply 
brute replication of DNS infrastructure. Along the lines we 
argued in §2, the c-infrastructure helps reduce the number 
of replicas needed to achieve a latency bound - for example, 
within the contiguous US, just 7 replicas would be enough 
to serve most of the population within 5ms. An alternate 
approach is to try achieving ‘on-path’ lookups in a man¬ 
ner proposed by ASAP [62], where the client sends its data 
request itself to the DNS server, which resolves the desti¬ 
nation’s address and forwards the data request on behalf of 
the client. Another method of achieving ‘on-path’ lookups 
is name-based routing. However these latter methods have 
their limitations, requiring extensive architectural changes. 
Eliminating the TCP handshake: In the absence of a 
handshake, a malicious client may craft packets with a source 
address different from its own, thus causing the server to 
send the response to that address. This technique can be used 
to attack both the server and other end hosts (which may re¬ 
ceive large volumes of data from several servers directed to 
them by malicious clients). The handshake, by requiring that 
the client acknowledge a message with a nonce before any 
significant compute or network resources are spent by the 
server, prevents this. Avoiding the handshake thus entails 
address authentication by some other means. Several solu¬ 
tions are available, including ASAP [62], TCP Fast Open’s 
Cookie mechanism [53], and APIP [46], with the broad idea 
being that the server (or another accountable third-party) val¬ 
idates the client’s claim to the address (or, in APIP’s case, the 
data packets); subsequently, this validation is included with 
future requests thus eliminating the handshake for those re¬ 
quests. The latency cost of validation may be incurred much 
less frequently than with handshaking for each connection. 

Eliminating TCP slow-start: As we see in §4, TCP’s slow 
start mechanism implies that fetching even just a few tens 
of kilobytes requires several RTTs. A variety of possibilities 
exist for handing this problem, the simplest proposal being 
to use larger TCP initial window sizes [30]. In our scenario, 
one could consider sending a larger TCP window’s worth of 
data over the c-infrastructure, and another few tens of kilo¬ 
bytes (if necessary) across traditional connectivity. There 
are of course, other options, such as Jumpstart [43], which 
allows for any ‘appropriate’ rate from the beginning, where 
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the appropriate rate may be determined in several possible 
ways; for example, ISPs could maintain a per-prefix listing 
of suitable window sizes based on recent history, etc. An¬ 
other proposal addressing the slow-start problem, RC3 [44] 
suggests using multiple priority levels, sending out a small 
volume at the highest priority, and an exponentially increas¬ 
ing volume of data at successively lower priority levels, thus 
sending out a large volume of data right from the start, while 
removing the possibility of congestion collapse. (In RC3, 
packets at each priority obey TCP’s congestion collapse di¬ 
rectives.) RC3 does however pose deployment hurdles, re¬ 
quiring that all routers support multiple priorities. 

TLS/SSL: In addition to the various protocol pieces above, 
making secure connections quickly is also an important prob¬ 
lem. Google’s experimental QUIC protocol [8] may present 
a solution here. QUIC proposes that once a client has a 
server’s public key (perhaps through a prior contact, which 
does require an additional round-trip), it can send its data 
request, together with a proposed session key, all encrypted 
with the server’s public key. The server can accept the ses¬ 
sion key (in the common case) and encrypt the response with 
it. Of course, QUIC too requires the elimination of the hand¬ 
shake and slow-start, and how exactly each objective will be 
met is currently being worked out. 

QUIC’s modular design makes it a useful vehicle for pro¬ 
tocol improvements such as eliminating the TCP handshake 
and folding in a more aggressive congestion control algo¬ 
rithm. Encouragingly, Google’s servers already support it, 
and while it is at this stage experimental, with several pieces 
of the design yet to be frozen, clients using Google’s Chrome 
browser or Opera can enable it [8]. 

The c-network could also spur the development and use 
of new protocols, for instance, by using new protocols inter¬ 
nally and deploying proxies at the edge. This would be simi¬ 
lar to what Akamai’s SureRoute [17] does (in terms of main¬ 
taining persistent TCP connections between Akamai servers.) 

6. RELATED WORK 

There is a large body of work on reducing Internet latency. 
However, this work has been limited in its scope, its scale, 
and most crucially, its ambition. Several efforts have focused 
on particular pieces; for example, [53, 62] focus on TCP 
handshakes; [30] on TCP’s initial congestion window; [58] 
on DNS resolution; [45, 34] on routing inflation due to BGP 
policy. Other work has discussed results from small scale 
experiments; for example, [56] presents performance mea¬ 
surements for 9 popular Web sites; [38] presents DNS and 
TCP measurements for the most popular 100 Web sites. The 
WProf [59] project breaks down Web page load time for 350 
Web pages into computational aspects of page rendering, as 
well as DNS and TCP handshake times. Wang et al. [60] in¬ 
vestigate latency on mobile browsers, but focus on the com¬ 
pute aspects rather than networking. 

The central question we have not seen answered, or even 
posed before, is ‘Why are we so far from the speed of light?’. 


Even the ramifications of a speed-of-light Internet have not 
been explored in any depth — how would such an advance 
change computing and its role in our lives? Answering these 
questions, and thereby helping to set the agenda for network¬ 
ing research in this direction is one of our primary objectives. 

The 2013 Workshop on Reducing Internet Latency [14] 
focused on potential mitigation techniques, with bufferbloat 
and active queue management being among the centerpieces. 
One interesting outcome of the workshop was a qualitative 
chart of latency reduction techniques, and their potential im¬ 
pact and feasibility (Fig. 1 in [14]). In a similar vein, one 
objective of our work is to quantify the latency gaps, sepa¬ 
rating out factors which are fundamental (like the c-bound) 
from those we might hope to improve. The goal of achieving 
latencies imperceptible to humans was also articulated [57]. 
We share that vision, and in §2 discuss the possible impacts 
of that technological leap. 

Further, beyond staking out the problem and discussing 
the potential impacts of a speed-of-light Internet, our mea¬ 
surements and analysis put the focus on an aspect of the la¬ 
tency problem that has been largely ignored so far: infras¬ 
tructural inefficiencies. We have not seen any work from 
the community directed at tackling the infrastructural in¬ 
efficiencies that contribute to latency inflation. There are 
other ambitious projects related to enhancing Internet infras¬ 
tructure, like the satellite Internet efforts of OneWeb and 
WorldVu [19, 36], Google’s Loon project [5], and Face- 
book’s drones [40], but these are all addressing a different 
(albeit important) problem — expanding the Internet’s reach 
to under-served populations. There are also efforts geared 
at improving bandwidth in existing Internet markets, such 
as Google Fiber [4]. But infrastructural latency has so far 
only garnered attention in niche scenarios, such as the finan¬ 
cial markets, and isolated submarine cable projects aimed at 
shortening specific routes [48, 47]. We believe this to be the 
first proposal to tackle the role of infrastructure in latency 
at such a wide scale. In fact, we make the case here that in¬ 
frastructural latency inflation is not even well understood, let 
alone effectively addressed. Our work also shows it to be a 
surprisingly more tractable problem than one might believe. 

A recent workshop paper [55] also addressed the necessity 
for a speed-of-light Internet, and some basic analysis of the 
causes of latency inflation. This work sharpens the focus on 
infrastructural latency inflation, putting it in perspective, and 
proposes a solution to the problem. 

7. CONCLUSION 

Speed-of-light Internet connectivity would be a techno¬ 
logical leap with phenomenal consequences, including the 
potential for new applications, instant response, and radical 
changes in the interactions between people and computing. 
To shed light on what’s keeping us from this vision, in this 
work, we quantify the latency gaps introduced by the Inter¬ 
net’s physical infrastructure and its network protocols, and 
find that infrastructural gaps make as significant a contribu¬ 
tion to latency inflation as protocol overheads. Further, we 
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propose a surprisingly cost-effective method of building a 
nearly speed-of-light Internet infrastructure. 
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